INSEAD - Wharton Alliance Center for Global Research & Development

This paper presents a new approach to the simulation ranking and selection problem, one that maximizes the expected NPV of decisions made when using stochastic simulation. Our formulation assumes that facilities exist to simulate a fixed number of alternative projects, and we pose the problem as a “stoppable” version of a Bayesian bandit problem. We show that, under relatively general conditions, a Gittins index can be used to indicate which system to simulate or implement. We then provide an asymptotic approximation for the index that is appropriate when simulation outputs are normally distributed with known but potentially different variances for the different systems. We conclude with a discussion that relaxes some of those assumptions. Managers must decide the operating characteristics of their companies’ manufacturing, supply chain, or service delivery systems. Often the decision reflects the choice of one among a number of competing designs. System designs may differ in the numbers or types of resources employed, as well as operating procedures used. To aid their decision-making managers may use stochastic or discrete event simulation. Simulation represents a widely-used and relatively low cost ‘insurance’ mechanism to estimate the performance of alternative systems and to improve the chances that the best system is implemented. The simulation selection problem formalizes the tradeoffs to be made when using simulation to evaluate a fixed number of competing design alternatives. For a fixed set of k alternative designs, one must decide how long to simulate each alternative and, given the simulation results, which design to implement. The most common approach for selecting the best of a finite set of simulated systems uses ranking and selection procedures, and the last 10 years have seen exciting progress in the development of this approach. Kim and Nelson (2005b) describe an indifference-zone approach which seeks to provides a probabilistic guarantee that the system with the best performance is ultimately selected, given that the best system is at least some amount (δ∗ > 0) better than the others and that the procedure is applied repeatedly to the same